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This application is a continuation-in-part of, and claims priority from, U.S. 
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incorporated by reference in its entirety for all purposes. This application also claims priority 
from French Patent Application No. 01 02539, filed February 23, 2001, which is hereby 
incorporated by reference in its entirety for all purposes. 



BACKGROUND OF THE INVENTION 
The invention relates generally to methods and devices for automatic visual 
perception, and more particularly to methods and devices for processing image signals using 
two or more histogram calculation units to localize one or more objects in an image signal 
using one or more characteristics an object such as the shape, size and orientation of the 
object. Such devices can be termed an electronic spatio-temporal neuron, and are particularly 
useful for image processing, but may also be used for processing other signals, such as audio 
signals. The techniques of the present invention are also particularly useful for tracking one 
or more objects in real time. 

Methods and devices are already known that have suggested statistical 
analysis of the points or pixels of a digital video signal coming from an observation system, 
for the realization of efficient devices capable of operating in real time. PCT WO 98/05002 
relates to one such technique. 

It is desirable to provide devices including combined data processing units of a 
similar nature, each addressing a particular parameter extracted from the video signal. In 
particular, it is desirable to provide devices including multiple units for calculating 
histograms, or electronic spatio-temporal neuron STN, each processing a DATA(A), ... by a 
function (f Q g) in order to generate individually an output value S, wherein these output values 
form together a feedback R available on a bus. 




BRIEF SUMMARY OF THE INVENTION 
It is thus that the present invention provides systems and methods 
implementing multiple histogram calculation units for the calculation of histograms or 
5 electronic spatio-temporal neuron STN, each processing a DATA(A), ... by a function (f 0 g) 
in order to generate individually an output value S, wherein these output values form together 
a feedback R available on a bus 110. At the same time, each of these histogram calculation 
units feeds and updates an analysis output register reg a supplying statistical information on 
the corresponding parameter. The selection of the parameter processed by each histogram 
10 calculation unit, the content of the analysis output register and the function (f G g) that the 
histogram calculation unit applies, are determined by API (Application Program Interface) 
software. 

*% The present invention also provides a method for perception of an object using 

characteristics, such as its shape, its size and/or its orientation, using a device composed of a 
m 1 5 set of histogram calculation units. 

21 Using the techniques of the present invention, a general outline of a moving 

object is determined with respect to a relatively stable background, then inside this outline, 
p elements that are characterized by their tone, color, relative position, etc. are determined... 
S% The present invention provides for multiple applications involving the 

GO 20 perception of an object. Such applications can be developed, either from a previous 
iU formalization having underlined the significant characteristics of the object or based on a 
learning function by examining the scenery in which the object in question is present, 
wherein such a device provides for automatic extraction of the characteristic parameters of 
the object. 

25 According to the present invention: 

a) a region of interest of the space is perceived in relation to a statistic 
criterion applied to a temporal parameter, 

b) the main region thus perceived is deactivated, 

c) a) and b) are repeated in order to perceive other regions of interest 
30 inside a non-deactivated space region, 

d) the procedure is stopped when a remaining region, non-deactivated, in 
the space does not provide a region of interest corresponding to the statistic criterion, 

e) a counter is incremented by consecutive valid frame, for each region of 
interest thus perceived, the center of gravity of its scatter chart, 
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f) for each region of interest thus perceived, the center of gravity of its 
scatter chart is captured. 

This invention also relates to the characteristics as will be evident by the 
following description and which should be considered individually or in all their possible 
5 technical combinations: 

- a counter is associated with each region of interest, and the counter value is 
incremented by one unit at each of the successive frames from which the said region of 
interest is perceived, whereas the value of this counted is reset to the first frame for which the 
said region is not perceived any longer, 

10 - the position of the center of gravity of the points making up a region of 

interest are stored in an associated memory thereby enabling its identification, 

- the region of interest is validated for one value of its associated counter that 
is greater than 1 , 

- the validated region is perceived by its center of gravity, the orientation of 
15 its projection axes and the sizes of the associated frame, 

- inside the region of interest, one or several secondary regions defined by 
one or several selection criteria are registered, 

- a secondary region plays the part of the region of interest, which leads to 
registering tertiary regions, 

20 - localization of the secondary regions is used for tracking the movements of 

the main region, 

- the temporal parameter is velocity, 

- the temporal parameter is a luminance level, 

- the temporal parameter is a color, 

25 - the temporal parameter is spatial resolution, 

- the temporal parameter is field depth, 

- the registered region is defined with respect to a mark selected among 
several marks of different orientations, 

- the relative positions of the centers of gravity of the scatter charts of the 
30 regions of interest registered serve for controlling the shape of the object perceived, 

- the shape is a human face, 

- the main region is the face and secondary regions are selected among the set 
encompassing the eyes, the mouth, the eyebrows and the nose. 
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The invention also relates to a device for localization of a shape in a space 
represented by pixels forming together a multidimensional space i, j, evolving with time, and 
represented at a succession of instants T, wherein the data each associated with a temporal 
parameter A, B, ... is in the form of digital signals DATA(A), DATA(B),.. .composed of a 
5 sequence Aij t , Bj Jt , ... of binary numbers of n bits associated with synchronization signals 
enabling to define the instant T of the space and the position i, j in this space, at which the 
signals Aj Jt , By t ,... were received at a given instant. 

Such a localization device according to the present invention typically 
comprises two sets of histogram calculation units receiving the signals and each generating a 
10 classification value, wherein the first unit receives a signal carrying a first temporal parameter 
and the second unit receives two spatial signals, wherein the classification value of the first 
unit validates a group of points in space that are processed by the second unit, the number of 
points being ni, the classification value of the second unit validating the parameter values 
CO processed by the first unity, wherein the two units generate jointly a binary signal ZA 
gil5 representing a region of interest and a signal P representing the value of the temporal 
21 parameter in this region. 

L J3 Such a device also typically comprises a third unit receiving a signal carrying 

0 a second temporal parameter, wherein the third unit operates similarly to the first unit and 

1 replacing the first unit when validating space points whose number is 112, where r\2 is greater 
CO 20 thanni. 

l7 Such a device also typically comprises several unit receiving space signals 

enabling successive validation of several groups of space points. 

Such a device also typically contains a set of histogram calculation units 
operated by an API software and connected together by a data bus and by a feedback bus. 

25 Reference to the remaining portions of the specification, including the 

drawings and claims, will realize other features and advantages of the present invention. 
Further features and advantages of the present invention, as well as the structure and 
operation of various embodiments of the present invention, are described in detail below with 
respect to the accompanying drawings. In the drawings, like reference numbers indicate 

30 identical or functionally similar elements. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The invention will be described in detail with reference to the appended 
drawings in which: 
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- Figure 1 is a representation of the histogram calculation unit according to the 
invention, in its context: 

- Figure 2 is a representation of the input video signal, processed by the device 
and the method of the invention and of the control signals generated by a sequencer; 

5 - Figure 3 is a diagram representing a passive histogram calculation unit; 

- Figure 4 is a diagram representing a self-adapting histogram calculation unit 
according to the invention with the anticipation and learning functionalities; 

- Figure 5 is a diagram representing signals processed by the calculation unit 

of Figure 4; 

10 - Figure 6 is the flow chart of the software controlling the calculation unit of 

Figure 4 in master mode; 

- Figure 7 is the flow chart of the software controlling the calculation unit of 
Figure 4 in slave mode; 

- Figure 8 is the flow chart of the insertion software of the curve zone; 

15 - Figure 9 is the flow chart of the initialisation software (generation of the 

command C INIT 5 ); 

- Figure 10 is the flow chart of the statistical calculation software (use of the 
command 'WRITE'); 

- Figure 1 1 is a flow chart of processing end (use of the command 'END 5 ); 

20 - Figure 12 is a representation of the elements of the histogram calculation unit 

with a self-adapting functionality according to one embodiment of the present invention; 

- Figures 13 and 13c are representations of an enabling counter fitted with 
several adapting modules according to alternate embodiments of the present invention; 

- Figures 13a and 13b are representations of statistical distributions of a 
25 parameter and classification criteria; 

- Figure 14 is a representation of the elements of histogram calculation unit 
producing POSMOY values according to one embodiment of the present invention; 

- Figure 15 is a diagram representing the elements of a self-adapting histogram 
calculation unit with anticipation according to a first embodiment; 

30 - Figure 15a is a diagram representing the elements of a self-adapting 

histogram calculation unit with anticipation according to an alternate embodiment; 

Figure 16 is a diagram of the classifier memory according to one embodiment 
of the present invention; 
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- Figure 17 is a diagram representing the elements of the self-adapting 
histogram calculation unit with anticipation according to a alternate embodiment; 

- Figure 18 is a detailed representation of the classifier memory with a bit- 
operated elementary calculation automaton according to one embodiment of the present 

5 invention; 

- Figure 19 is a representation of an elementary anticipation calculation 
automaton according to one embodiment of the present invention; 

- Figure 20 is a schematic representation of the anticipation process according 
to one embodiment of the present invention; 

10 - Figure 21 is the flow chart of the anticipation implementation software 

according to one embodiment of the present invention; 

- Figure 22 is a representation of the time coincidences unit according to one 
~ embodiment of the present invention; 

00 - Figure 23 is a flow chart representation of a field programmable gate array 

fp 15 (FPGA) used as a time coincidences unit according to one embodiment of the present 
J; invention; 

£ s - Figure 24 is the register-based representation, limited to one row of the 

q system, o f Figure 2 3 ; 

zJ - Figure 25 is a representation of the elements of a histogram calculation unit 

05 20 with a learning functionality according to one embodiment of the present invention; 
G. - Figure 26 is a schematic representation of axis selection circuitry according 

to one embodiment of the present invention; 

- Figure 27 illustrates various axes selectable by the circuitry of Figure 26; 

- Figure 28 is a schematic representation of a statistical visualisation device 
25 according to one embodiment of the present invention; 

- Figure 29 is an example of the result obtained using the visualisation 
produced by the device of Figure 28; 

- Figure 30 is the representation of an implementation of a number of 
histogram calculation units according to one embodiment of the present invention; 

30 - Figure 3 1 is the representation of the use of a single programmable 

histogram calculation unit with a multiplexer enabling the calculation unit to process any of a 
number of parameters according to one embodiment of the present invention; 

- Figure 3 1 a is a representation of a histogram calculation unit called as well 
an electronic spatio-temporal neuron; 
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- Figure 32 represents a set of histogram calculation units with programmable 
input control in their context of usage thereby constituting a functional entity according to 
one embodiment of the present invention; 

- Figure 33 is a synthetic representation of a functional unit with an associated 
5 signal generator according to one embodiment of the present invention; 

- Figure 34 corresponds to Figure 32 in the case of a two-source acquisition; 

- Figure 35 corresponds to Figure 33 in the case of a binocular acquisition; 

- Figure 36 is a schematic representation of a signal generator fitted with 
controlled optics according to one embodiment of the present invention; 

10 - Figure 37 shows the case of a three-source acquisition according to one 

embodiment of the present invention; 

- Figure 38 is a representation of the application management interface (API) 
y according to one embodiment of the present invention; 

CO - Figure 39 illustrates a system for processing signals in the sound perception 

fp 15 domain according to one embodiment of the present invention; and 

;r! - Figure 40 is a simplified representation of a device according to an 

ffi embodiment of the present invention. 

q - Figure 41 is the representation of a histogram calculation unit according to 

S; the present invention; 

W 20 - Figure 42 illustrates a device including two histogram calculation units 

f7 fulfilling a spatial compound function according to the invention; 

- Figure 43 is an alternate representation of the device of Figure 42; 

- Figure 44 illustrates a set of histogram calculation units used for the 
detection and localization of an object in black and white; 

25 - Figure 45 illustrates a device including a set of histogram calculation units 

used for the detection and localization of an object in color; 

- Figure 46 is an alternate representation of the device of Figure 45 ; 

- Figure 47 illustrates a system including the combination of the device of 
Figure 43 and a histogram calculation unit applied to movement; 

30 — Figure 48 represents the localization of a region of interest; 

- Figures 49 A, B and C illustrate representations of the histograms, 
respectively, of the movement, of a first spatial parameter (X) and of a second spatial 
parameter (Y) for the region of interest of Figure 48; 

- Figure 50 is a localization flowchart of a region of interest; 
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- Figure 51 represents a device configured to localize three regions of interest 



from a main region; 



- Figure 52 illustrates the operation of the device of Figure 51 in the form of a 



graph; 
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- Figure 53 is a representation of the analysis of the shape formed by a human 



face; 

- Figure 54 illustrates the use of the results produced by the shape analysis, 
wherein these results can be transferred remotely and the image reconstructed. 
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DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 



The invention is subject to numerous embodiments. The information processed can * 
be of various natures and represent multiple data or parameters. However, one application is 
image processing, wherein the images make up the space considered. This space in one 
15 embodiment is two-dimensional. The following detailed description corresponds to this 

particular embodiment. It should be appreciated, however, that the space may be of three or 
more dimensions. 

A histogram calculation unit 1 of the invention is represented in its context by Figures 
1 and 2. Histogram calculation unit 1 is part of a perception unit 13 that receives and 

20 processes a signal S(t) or S(PI). The histogram calculation unit 1 processes and generates 

time coincidences information S'(t) on a bus 111. More precisely, Figure 1 represents several 
associated histogram calculation units 1 A, IB, . . ., IE in the same perception unit. In one 
embodiment, perception unit 13 is a visual perception unit that processes various signals 
relating to a visual scene or scenes. In other embodiments, the perception unit 13 processes 

25 signals related to the desired perception parameters, for example, sound parameters. The 

following will discuss the invention with respect to the visual perception domain, although it 
will be apparent that other perception domains may be implemented. 

A sequencer 9 generates, out of the synchronisation signals ST, SL, CLOCK, 
sequence signals INIT, WRITE and COUNTER that control the histogram calculation unit. 

30 As represented on Figure 1, the input signals of the sequencer 9 (St, SL, ST, CLOCK) 

may come from a signal generator assembly 2 comprising a camera 22 or a signal generator 
assembly 3 comprising a CMOS imaging device 32. It will be apparent that input signals can 
be supplied by any signal generation mechanism. 
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When the input signals come from an assembly 2 comprising a camera, this assembly 
imposes frame and line synchronisation signals so that the histogram calculation unit and its 
sequencer operate in a slave mode or synchronisation slave mode. Figure 7 illustrates a flow 
chart representing software for controlling the histogram calculation unit and sequencer in a 
5 slave mode. 

Conversely, in case when these signals come from an assembly 3 comprising a CMOS 
imaging device, the sequencer 9 operates in a master mode and generates itself the 
synchronisation signals. Figure 6 illustrates a flow chart representing software for controlling 
the histogram calculation unit and sequencer in a master mode. 

10 More precisely, the assembly 2 enables acquisition of data from a scene 21 by a 

camera 22. The camera 22 produces a signal S(PI) whose configuration, of the type 
represented on Figure 2, will be described in detail below. 

The electronic control unit 23 of the camera 22 then provides the signals S(t) resulting 
from the extraction of S(PI), ST, SL synchronisation signals and the CLOCK signal 

15 originating from a phase-lock loop, that are used by the histogram calculation unit. 

In the case of an assembly 3 comprising a CMOS imaging device, this imaging device 
32 is used for the acquisition of data of the scene 31, it supplies S(t) and is driven by a 
synchronization unit 33 that produces the frame synchronization signals ST and the line 
synchronization signals SL, as well as the CLOCK signal used by the CMOS imaging device 

20 32 as well as by the other elements of the visual perception unit 13. 

The histogram calculation units 1 are advantageously co-ordinated to a spatial 
processing unit 6 and a temporal processing unit 5 and to a delay line 7 that have been 
described in FR-2.61 1063, and FR-2.75 1.772, and WO-98/05002, and corresponding U.S. 
Patent Application Serial No. 09/230,502, entitled "Image Processing Apparatus and 

25 Method," filed on January 26, 1999, the contents of which are each hereby incorporated by 
reference in its entirety for all purposes. The spatial and temporal processing units 5 and 6 
correspond to the device referred to as 1 1 in the patent application mentioned. It receives the 
signal S(PI) and generates parameters V (speed), DI (direction), each corresponding to one of 
the inputs identified as DATA(A) . . . DATA(E) in this application. 

30 In one embodiment, these parameters include the spatial resolution, the image 

structure (multiscale contrast change in polar coordinates, etc. . . .), as they result from a 
wavelet analysis by Gabor and described in Daugman's article (1988) "Complete Discrete 2D 
Gabor Transform..., IEEE Trans. Acoust. Speech Signal Process 36:1169-1179. 
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This assembly, composed of a plurality of histogram calculation units 1, the spatial 
and temporal processing units 5 and 6 and the delay line 7, supplies 'time coincidences' 
information, generally in digital form, that can be processed by a downstream device, or a 
signal enabling visualisation of information on a screen 8 via the bus 111. 
5 A passive (non self-adapting) histogram calculation unit and without anticipation is 

represented on Figure 3. This histogram calculation unit is intended for processing the values 
of a parameter A that are affected at each pixel in a signal S(t) = {a^r} of the video type. 

However, it will be apparent that the system is capable of processing values 
associated with signals other than video signals. 

10 Video signal S is composed of a succession of frames, wherein each frame includes a 

succession of pixels whose assembly forms a space, for example an image for a two- 
dimensional space. In such a case, the frames are themselves broken down into lines and 
columns. This signal S(t) carries a value of the parameter A for each pixel (i,j). The 
succession of the frames represents therefore the temporal succession of images. In the 

15 notation {ajj T }, T represents the frame, i is the number of a line in the frame T, j is the number 
of the column of the pixel in this line, a is the value of the parameter A associated with the 
pixel ijT. 

The signal S can be an analogue signal. However, it is preferably digital and 
composed, as represented on Figure 2, of a succession of frames Ti and T 2 , each being 
20 formed of a succession of horizontal scanned lines such as Ii.i, Ii. 2 , . . .» I\m for Ti and I 2 .i . . . 
for T 2 . Each line includes a succession of pixels or image points PL 

S(PI) comprises a frame synchronisation signal (ST) at the beginning of each frame, a 
line synchronisation signal (SL) at the beginning of each line that is not a beginning of frame 
as well. Thus, S(PI) comprises a succession of frames that represents the temporal array and, 
25 within each frame, a series of lines and of pixels arranged in columns that are significant of 
the spatial array. 

In the temporal array, «successive frames» designate chronologically successive 
frames and «successive pixels at the same position» designate the successive values aij 
associated respectively to the pixels (i j) placed at the same location in the successive frames, 
30 i.e. for instance (1, 1) of Ii.i in the frame Ti and (1,1) of I 2 .i in the corresponding following 
frame T 2 ... 

On the basis of the S(PI), as indicated above with reference to the application 
PCT/FR-97/01354, the spatial and temporal processing units 5 and 6 generate one or a 
plurality of signals, e.g., signals DATA(A) . . . DATA(E). 
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The passive histogram calculation unit 1, without anticipation, as represented in 
Figure 3, processes a signal DATA(A) whose structure is represented in Figure 2. This signal 
may be received directly from either a camera or any other image acquisition system, or may 
have been subjected previously to a first process, for example spatial and/or temporal 
5 processing as described, for example, in application PCT/FR-97/01354, the contents of which 
is hereby incorporated by reference in its entirety for all purposes. 

Classifier 101 generates a signal 101s of similar structure that carries for each pixel a 
piece of information significant of the result obtained when applying recognition or selection 
criteria. 

10 As shown in Figure 3, histogram calculation unit 1 includes an analysis 

memory 100, an address multiplexer 105, a data input multiplexer 106, an incrementation 
unit 107 classifier 101, a time coincidences unit 102 and a test unit 103, the operations of 
which will be described below. All elements of the histogram calculation unit 1 are 
controlled and synchronised by a clock signal (not shown). 



I. The Analysis Memory 

Analysis memory 100 is preferably a conventional synchronous or asynchronous 
digital memory, such as a DRAM, SDRAM or the like. Analysis memory 100 includes a 
number, n, of addresses, d, equal to the number of possible levels for the values of the 

20 parameter A that must be discriminated. Each of these addresses preferably stores at least the 
number of pixels contained in a frame (i.e., in an image). 

For each frame, after resetting by the command signal EMIT, a signal WRITE enables, 
throughout the frame, the processing of the data DATA(A). Thus, the analysis memory 100 is 
capable of receiving the signal DATA(A). For each frame received, the pixels for which the 

25 value of the parameter A has a value = d (if they are enabled by an enabling signal 102s as 
will be described below), will increment the content of the address of row d of the memory 
100 by a value 1. Thus, after having received a complete frame, the memory 100 comprises, 
at each of its addresses d, the number of pixels that are enabled and for which the parameter 
A had a value d. 



II. The Address and Data Input Multiplexers 

The histogram calculation unit 1 also comprises an address multiplexer 105 and a data 
input multiplexer 106. Each multiplexer receives a binary selection control signal 
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and two input signals and provides one output signal. The value of the output of each 
multiplexer corresponds to one of the inputs when the selection control signal is equal to one 
value, e.g., 1 5 and the other input when the control signal is equal to a different value, e.g., 
zero as shown. 

5 As shown, when the control signal EMIT is equal to zero, the address multiplexer 1 05 

selects an address in the analysis memory 100 in relation to the level d of the signal received 
(ajjT), and the data input multiplexer 106 transfers the incrementation of the value contained 
in this memory from incrementor 107 in relation to the status of validation signal 102s. 

As shown, when the control signal INIT is equal to 1, the address multiplexer 105 
1 0 transfers the signal of the counter that increments the address from zero to the maximum 
value of DAT A( A), and the data input multiplexer 106 forces zero on the input of the 
memory 100. 

2 III. The Incrementation Unit 

=J15 Incrementation unit 107 in one embodiment is a controlled adder comprising one 

J input, one enabling signal input and one output. 

1 The output of the incrementation unit is equal to the output of the analysis memory 

v 100 if the enabling signal 102s is equal to one value, e.g., zero; or it is equal to output of the 

i analysis memory 100 increased by 1 if the enabling signal 102s is equal to another value, e.g., 

j20 1. 

IV. The Classifier 

Classifier unit 101 includes a register lOlr capable of storing certain possible level 
values (di, d 2 , . . .) for the levels of the parameter A. 
25 The classifier 101 receives the signal DATA(A), sorts the pixels, and provides, on its 

output 101s, a value 1 when the parameter A associated with the said pixel has a level 
corresponding to that contained in the register lOlr (di, 62, ...) and the zero value conversely. 

The output of the classifier 101 is connected to a bus 111. 

30 V. The Time Coincidences Unit 

Time coincidences unit 102 is connected to the bus 111, and includes at least one 
register 102r and receives, for each pixel, the output values (in E , . . . , in B , in A ) of the classifiers 
101 of the various histogram calculation units 1 connected to the bus 111. 
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This time coincidences unit 1 02 compares the values thus received to those contained 
in its register 102r and transmits, on its output 102s, for each pixel, an enabling signal equal 
to 1 when there is a coincidence between the register values equal to 1 and the corresponding 
data received from the bus 111, and a zero value in the reverse case, which corresponds to the 
5 following Boolean function: 



VI. The Test Unit and the Analysis Output Register 

Test unit 103 receiving the information coming from the analysis memory 100 and is 

10 connected to analysis output registers 104. The analysis output registers 104 are intended for 
receiving statistical information prepared on the basis of the values of the parameter A of the 
signal DATA(A) for each frame. This information may be, for example, minimum values 
(MIN) and maximum values (MAX) of the parameter A, of the number of occurrences 
(RMAX) of the most represented value and of the position (POSRMAX) of this value, as 

1 5 well as of the number (NBPTS) of points for which information has already been received. 

The test unit 103 updates the analysis output registers 104 in relation to the information that it 
receives. 

The incrementation enabling unit 1 07 also outputs a signal addressed to the test unit 

103 that enables test unit 103 to increment the analysis output register 104. 

20 After processing a complete frame, the histogram calculation unit 1 has produced 

statistical information representative of this frame, available in the analysis output register 

104 and processable for all intended purposes, either for operator-accessible visualisation or 
for processing by any other programme or automaton. 



25 such as the minimum (MIN) of the histogram, the maximum (MAX) of the histogram, the 
number of points (NBPTS) of the histogram, the position (POSRMAX) of the maximum of 
the histogram and the number of points (RMAX) at the maximum of the histogram. These 
features are determined in parallel with the formation of the histogram by the test unit 103, as 
follows: 

30 For each pixel that is validated: 



out = (in 0 + Reg o ).(in 1 +Reg 1 )...(in n +Reg n Xin o +in 1 + ...in n ) 



The analysis output registers 104 comprise memories for each key-feature 
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(a) if the value of the parameter DATA(A) of the pixel < MIN (which is initially set to 
the maximum possible value of DAT A( A) of the histogram), then the value of the parameter 
is inscribed in MIN; 

(b) if the value of the parameter DATA(A) of the pixel > MAX (which is initially set 
5 to the minimum possible value of DAT A( A) of the histogram), then the value of the 

parameter is inscribed in MAX; 

(c) if the content of the memory 100 at the address of the value of the parameter of the 
pixel > RMAX (which is initially set to the minimum possible value DATA(A) of the 
histogram), then i) the value of the parameter should be written into PORMAX and ii) the 

10 output of the memory should be written into RMAX; 

(d) NBPTS (which is initially set to the value zero) should be increased by one unit. 
Figure 10 illustrates a flow chart of the statistical calculation software according to 

one embodiment of the invention. 



15 VII. Global Operation of Passive Histogram Calculation Units 

According to one embodiment of the invention, a plurality of histogram calculation 
units, e.g., 1 A, IB, . . IE, are connected to the same time coincidences bus 111. This 
description refers to five histogram calculation units A to E, although extrapolation to any 
number of units is evident. 

20 A. Signal WRITE 

For each signal WRITE, the classifier unit 101 of each histogram processing unit 1 
supplies to the bus 1 1 1, for each pixel, an output signal 101s and each of the histogram 
processing units 1 receives all these signals on the input in A , . . in E of their time 
coincidences unit 102. 

25 The parameter, for example DATA(A) for the unit 1 A, is compared to the content of 

the register lOlr of the classifier 101. The result, inA = lOlr, of this comparison is a binary 
signal 101 s that is addressed at the same time as its counterparts inB . . . inE, coming from the 
other units, e.g., IB ... IE. 

Each time coincidences unit 102 compares each of these values received to the 
30 content of its register 102r constituting a time coincidences criterion, R, and generates, on its 
output 102s, a binary signal whose value depends on the result of the comparison. 

This signal 102s controls the incrementer 107. For example, when signal 102s is 
equal to 1, incrementor 107 produces, via the data multiplexer 106, the incrementation by one 
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unit of the content of the register of the memory 100 corresponding to the value of the 
parameter DATA(A), simultaneously the test unit 103 ensures statistical processing of the 
content of the memory 100 and transfers the content thereof into the analysis output register 
104. 

5 At the end of the signal WRITE, each register of the memory 100 contains as a value 

d, the number of pixels for which the signal DATA(A) showed the corresponding value d and 
that the time coincidences unit 102 has enabled. 
B. Signal INIT 

During the signal INIT, the signal COUNTER that scans the values from 0 to n 
10 (number of addresses in memory 100), resets the registers of the memory 100. Figure 9 is a 
flow chart of initialisation software (e.g., generation of commend 'INIT') according to an 
embodiment of the invention. 

VIII. Self-adaptation 

15 In the description made until now, the memory of the classifier 101 includes a register 

lOlr whose content determined outside the system is fixed. Such a classifier is said to be 
"passive". 

According to one embodiment of the present invention, a self- adapting histogram 
processing unit 1 is provided. In this embodiment, the content of the memory of the classifier 

20 101 is automatically updated. In one embodiment, classifier 101 includes a look up 

table (LUT). To fulfil the self-adapting function, i.e. real time updating of the classifier 101, 
the histogram calculation unit 1 of Figure 3 is perfected in accordance with Figure 4. Instead 
of having a simple register lOlr written outside the system, the classifier 101 has an 
addressable memory whose writing is controlled by a signal END. The sequencer 9 

25 generates this signal END represented in Figure 5. The histogram calculation unit 1 

comprises a selection circuit 110, e.g., an 'OR' gate as shown, receiving at its input the 
signals INIT and END and whose output is connected to the selection input of the address 
multiplexer 105. 

The memory of the classifier 101 is controlled by the system, andits content is 
30 modifiable. In one embodiment, the classifier memory comprises a data input terminal for 
receiving a DATA IN signal, a write input terminal for receiving a write command WR 
signal, and an address input for receiving an ADDRESS signal. The address input is 
connected to the output of an anticipation multiplexer 108. This 'two to one'-type 
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multiplexer 1 08 comprises an anticipation control input terminal connected to the output of 
an operator 'OR' 112 receiving as its input the signals INIT and END. The inputs of the 
anticipation multiplexer receive the same signals as the inputs of the address multiplexer 105 
(e.g., DATA(A) and COUNTER). When the signal END is equal to 1, the memory of the 
5 classifier 101 is written by the signal resulting from the comparison between the value of the 
histogram memory 100 and a value derived from the analysis output register 104, e.g., 
RMAX/2, for the possible values of DAT A( A) as will be described in more detail below. 

Hence the classifier acts as a classification function f A which is the relationship that it 
establishes between the data DATA(A) that it receives and the output binary value (101s) A 
10 that it produces, via the memory of the classifier. 

A. First embodiment of classifier 
With reference to Figure 12, the classifier 101 fulfilling the self-adapting function 
comprises a memory 118 whose writing input terminal WR receives the signal END and the 
15 address input terminal ADDRESS receives the output signal of the address multiplexer 108. 

Classifier 101 also includes a comparator 119 comprising two inputs and one output 
that is connected to the data input DATA IN of the memory 118. 

The first input, Q, of the comparator 119 receives a value derived from an analysis 
output register 104 and its second input receives the output of the memory 100. In one 
20 embodiment, for example, the value received at the first input Q is the value RMAX/2, 
derived from RMAX register 104 in conjunction with divider circuit 121. 

Memory 1 18 of the classifier 101 preferably comprises the same number of words as 
the analysis memory 100, but in the memory 118, each word comprises one bit only. 

At the end (e.g., signal END = 1) of the reception of a new data flux DATA(A) of a 
25 given frame, a writing sequence starts. 

If for a given memory address d of the analysis memory 100, the value read is greater 
than RMAX/2, a value 1 is inscribed into the memory 118 at the corresponding position. 
Conversely, if the value read is less than RMAX/2, the value 0 is inscribed in this position. 
All the memory addresses d are scanned from 0 to n. The memory 1 18 of the classifier 101 is 
30 thus updated. It will be understood that values from registers 104 other than RMAX may be 
used. 

35 
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B. Second embodiment of classifier 
Figure 13 represents an alternative embodiment of the classifier 101 including a 

multiplexer 120 that is controlled by a selection control signal 124. Classifier 101 enables 

comparison of the parameter P to a statistical value Q, which can be prepared in various ways 

5 in relation to the statistical parameters received on the different inputs 0, 1, 2, 3 of 

multiplexer 120, which are selected by the selection control signal 124, which depends on the 

content of the register 'SELECTION'. The input 0 of the multiplexer 120 receives the value 

RMAX/2 produced on the basis of the data in the analysis output register 104 by the divider 

circuit 121, the input 1 of the multiplexer 120 receives directly the value RMAX, the input 2 

10 of the multiplexer 120 receives a threshold value contained in a register 'THRESHOLD' 123 

whose content is programmed outside the system, and the input 3 of multiplexer 120 receives 

the quotient of the number of points NBPTS by the THRESHOLD 123 produced by the 

divider circuit 122. 

Therefore, as represented on Figure 13, the parameter P can be compared to the 
15 respective values RMAX/2, RMAX, at a threshold B input from the outside and in proportion 
to the number of points NBPTS attached to this threshold by the divider 122. It will be 
apparent that other input values may be used, e.g., any values from registers 104. 

The content of the memory 1 18 is updated, in relation to the signals supplied by the 
comparator 119 similarly to the update described in the first embodiment. 

20 

C Third embodiment of classifier 

Figures 13a, 13b, 13c represents another embodiment of a classifier in which the 
cumulative total of events is used in a histogram instead of the levels. The classification 
boundaries are defined, for example, by the use of a register RMAX, corresponding to a 
25 maximum of events for the analyzed parameter, and in searching for the parameter values for 
RMAX/2. On both sides of the RMAX position, these values correspond to limit A and limit 
B of the classifier. 

Hence, the RMAX register such as it is operated in the second embodiment of the 
classifier, is replaced here by the register NBPTS, corresponding to the total cumulative 
30 result of events (Figure 13a). By removing a percentage k of NBPTS on both sides of the 
histogram, the limits A and B become more stable (Figure 13b). 

The device represented in Figure 13c carries out this function. 
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In Figure 13c, the analysis memory 100 and the command of the address multiplexer 
105 are present. The analysis output register 104 operates as described above using the 
number of points NBPTS 1041 and, in general, the limit A 1042 and the limit B 1043 as well. 

The learning register 117 receives the output data of the analysis memory 100 and 
5 supplies, via the register 301 fed by the adder 300, the inputs of two comparators 1151 and 
1 152, respectively, of the enabling calculator 115, which includes a memory 1 153 storing the 
value k, percentage of the number of points to take into consideration. 

A multiplier 1 154 receives the number of points NBPTS on one of its inputs and the 
value k on the other, feeds, on one side, the second input of the comparator 1151 and, on the 
10 other side, one of the inputs of a subtracter 1 155, which receives on its other input the 
number of points. The output of this subtracter 1 155 feeds the second input of the 
comparator 1152. 

The subtracter output 1 155 supplies the limit A, the comparator output 1 152 supplies 
the limit B and an operator "NON-AND" 1 156 receiving on each of its inputs, respectively 
15 the value of the limit A and on the inverted input, the value of the limit B, supplies the output 
signal of the enabling calculator 115. 

At the end of the histogram calculation, the register NBPTS is known and a signal 
MATRIX-END allows to know the value a = k , NBPTS and a value p = NBPTS - a. 

In initializing to zero a cumulative function S, that is to say S Q = zero, the increment i 
20 of a counter connected to the address of the previously determined histogram memory allows 
to reach the contents of this memory and to supply the cumulative register Si. 

A first test includes assigning to limit A, the increment value i as long as Sj is smaller 
than the previously defined a value. 

A second test includes assigning to limit B, the increment value i as long as Si is 
25 smaller than the previously defined p value. 

Generally, the classifier may be achieved according to numerous embodiments, the 
essential being that it allows to place the parameter DATA(A) with respect to values or limits 
statistically determined over a set of former data DATA(A). 

30 IX. The Memory 118 of the Classifier 101 

Figure 16 is a detailed representation of the memory 118 including an input 
demultiplexer with input enabling function 130 and an output multiplexer 131. The input 
multiplexer 130 receiving the writing signal WR is then capable of enabling the choice of the 
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register of the memory 118, selected by the address command ADDRESS, for writing the 
binary value of the comparison DATA IN. The output multiplexer 131 addresses the value of 
a particular register, selected by the address command ADDRESS, on the output 101s of the 
memory 1 1 8 of the classifier. 
5 The input demultiplexer 130 and the output multiplexer 131 are controlled via bus 

134 originated from the anticipation multiplexer 108. 

More precisely, the 1/n input demultiplexer 130, controlled by the address transmitted 
by the bus 134, sends the signal WR (WRITE), respectively in the form of the signals Sel 0 , 
Seli,Sel 2 , Sel n on the registers 140 0 , 140j, 140 2 , 140 n of order 0, 1, nand 
10 determines which of its registers as addressee of the content of the information transmitted by 
the signal DATA IN. The information originating from these registers 140 0 , 140i, 140 2 , . .., 
140 n is sent to the multiplexer 131, which determines the output, OUT. 

yn X. Anticipation 

— 15 In a preferred embodiment, in addition to real time updating of classifier 101, the 

}=3 histogram processing unit 1 is configured to perform an anticipation function. Such 
J3 anticipation of the self- adapting function of the classifier 101 improves the operation of this 

looped system and assimilates it to the operation of a biological system. The purpose of the 
P anticipation is to anticipate the value contained in the memory 1 18 of the classifier 101 in 
20 order to speed up the processing and thereby to facilitate the tracing of an object or of its 
ff evolution. 

To this end, the global variation of the histogram is calculated and the result is then 
used to apply the anticipation according to the following methods. In either case, the 
anticipation defines an anticipation function (f A og A ), linking the data DATA(A) and the value 
25 (1015) A , characterizing the histogram calculation unit processing the parameter A. 

A. Calculation of the global variation of the histogram 

Referring back to Figure 4, the test unit 103 and the analysis output registers 104 
generate a statistical value POSMOY whose values POSMOY 0 and POSMOYi for two 
30 successive frames are memorised. POSMOY is the value of a parameter, e.g., DATA(A), in 
relation to which, in a given frame, the parameter has a value greater than or equal to half the 
enabled points in the frame. 
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When the signal END is equal to 1 , the new value POSMOY 0 is calculated and the 
previous value of POSMOY 0 is saved in POSMOY,. 

With reference to Figure 14, the preparation of the variable POSMOY 0 will now be 
described.The variable POSMOY 0 is produced by a comparator 302, which on one of its 
5 inputs, Q, the parameter NBPTS from register 104 that is divided by two by the divider 

circuit 303. The second input P of comparator 302 receives the output of a register 301 that 
is controlled by the initialisation INIT and the end END signals, which receives as input the 
output of an adder 300. Adder 300 receives at one input, A, the output value of the register 
301 and on its second input, B, the output value of the memory 100 that has been described 
10 previously. Thus, the register 301, reset initially, stores the cumulated content of the registers 
of the memory that are scanned by the signal COUNTER from zero to n. As long as this 
cumulated value is smaller than NBPTS/2, the value of the COUNTER is stored in 
POSMOYo. At the end of the cycle END, POSMOY 0 therefore contains the last value 
COUNTER for which the cumulated value is smaller than NBPTS/2. 

15 

B. Application of the histogram variation to the anticipation (first method) 

Figure 15 illustrates a circuit according to one embodiment that is configured to 
implement anticipation. The memory 1 1 8 is that described previously with reference to 
Figure 16. 

20 A calculation unit 310 with sign extraction capability supplies the values |POSMOY 0 

minus POSMOYi) and the sign of this difference. These parameters control a translator 311 
after reversal of the sign by the inverter 312. The value of the parameter supplying the 
memory 1 18 is thus offset by the value |POSMOY 0 minus POSMOYi | with respect to the 
passive operation, in the direction opposite the POSMOY variation calculated in the unit 310. 

25 Figure 15a illustrates a circuit according to an alternate embodiment that is configured 

to implement anticipation. In this embodiment, calculation unit 3 10a is similar to calculation 
unit 310, but with improved performance by providing different functionality with respect to 
the offset of the value of the parameter supplying memory 118. Calculation unit 310 of 
Figure 15 provides an offset determined by a function of the form y=x, where x is 

30 |POSMOY 0 minus POSMOYi | (P0-P1). Calculation unit 310 or 310a of Figure 15 can 

provide for an offset determined by functions of the form y=ax+b, where a (e.g., kl and k2) 
and b (e.g., cl and c2) are adjustable constants provided, for example, by an on-chip 
controller. It will, of course, be apparent that any other function of the POSMOY values can 
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be used as desired, such as y=ax 2 . In one embodiment, for example, a multiplexer unit can be 
implemented to receive as input to the two functions of POSMOY, namely kl*|P0-Pl|+cl 
and k2*|P0-Pl|+c2, and provides one as output based on the value of the control signal 



Clock" to control translator 311. 
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To further increase the range of classification, OR circuit 125 and delay circuit 126 



are optionally provided. Delay circuit is controlled by the same signal, "Clock", that controls 
multiplexer 127. The output values from memory 118 related to the two different offset 
functions are then provided to OR gate 125, the output of which is signal 102s with an 
improved classification range, and therefore improved anticipation characteristics. 



C. Application of the histogram variation to the anticipation (second method) 

Figure 17 illustrates a circuit that is configured to implement anticipation according to 
another embodiment. The memory 1 18 is represented in Figure 18. 

The general architecture of the memory 118 has been described above. A sequence 
15 for a given bit will now be described, wherein the sequence for the other bits are analogous. 
The elements common to Figure 16 bear the same reference numbers. 

The register 140i is associated with an input multiplexer 160i that receives on one of 
its inputs, the binary signal (DATA IN) output from the comparator 119 and on its other 
input, the output signal of the anticipation calculation unit 150i. The input multiplexer 160i 
20 is controlled by the signal ETD that also controls the writing. The writing command of the 
register 140i is connected to an OR gate 170i that receives, on one of its inputs, the signal 
ETD and on the other, a signal Sell. 

At the output of the register 140i, an anticipation calculation unit 150i receives as 
input the three output signals Qo, Qi and Q2, from the registers 140o, 140i, 1402 of order, 
25 respectively, 0, 1,2. Unit 150i is commanded by the signals SM, SP and T. In the units 
150o, 150], . . . 150 n , anticipation is performed by the succession of expansion operations 
followed by erosion operations. 

An anticipation calculation unit 150 is described in detail on Figure 19. In one 
embodiment, unit 150 comprises a multiplexer 207 including one output and two inputs and 
30 is controlled by the signal T. One of the inputs of multiplexer 207 is connected to an 

expansion operator circuit 208, which supplies a signal Ai, and the other input is connected to 
an erosion operator circuit 209, which supplies a signal Bi- 
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The expansion operator circuit 208 comprises a three-input and one-output circuit 
201, which in one embodiment is an OR gate,, whose output is connected to the multiplexer 
207. The first input of circuit 201 is supplied by the signal Qi, the second input is supplied 
by the output from a two-input AND circuit 202, one of whose inputs is the signal Qo and the 
5 other input is the signal SP. The third input of the circuit 201 is supplied by the output of a 
two-input AND circuit 203, one of whose inputs is the signal Q 2 and the other the signal SM. 
The function fulfilled by the expansion operator 208 in this embodiment is thus: 

Ai = Qi + Qo x SP + Q 2 x SM. 

The erosion operator circuit 209 comprises a three-input and one-output circuit 204, 
10 which in one embodiment is an AND gate. The output is connected to the multiplexer 207. 

The first input of circuit 204 is supplied by the signal Qi, and the second input is connected to 
a four-input and one-output circuit NOT- AND circuit 205. The first input of NOT-AND 
circuit 205 is connected to the signal SP, the second to the signal Qi, the third input is 
connected to the signal Q 0 , and the fourth input is connected to the inverse of the signal Q 2 . 
15 A second NOT-AND circuit 206 has four inputs and an output connected to the third input of 
the AND circuit 204, wherein the first of these inputs is supplied by the signal Qi, the second 
by the signal SM, the third by the signal Q 2 and the fourth by the inverse of signal Qo. The 
function fulfilled by the erosion operator 209 in this embodiment is thus: 



20 B, = Q, x (SM x Q 2 x Q 0 ) x (SP x Q 2 x Q 0 ) 

An example of the anticipation operation is illustrated on Figure 20. In this Figure, on 
the left with reference to the time axis t, are represented the signals INIT, WRITE, END, 
ETD, T, SP, SM. The signal INIT, generated by the sequencer 9, starts the processing cycle 

25 of a frame. Throughout its duration, all the memories and registers are initialised. The signal 
WRITE, also generated by the sequencer 9, follows the signal INIT and controls the 
statistical calculations for the frame considered whose data is represented by the curve C, 
whose axes represent in abscissa the values of the parameter and in ordinate the number of 
occurrences. The test unit 103 looks for the maximum number of occurrences RMAX. 

30 At the end of the signal WRITE, the signal END, generated by the sequencer 9, 

enables the update of the memory of the classifier 118. The new data is generated by the 
comparator 119. Figure 1 1 illustrates a flow chart of software for updating the classifier 
memory according to one embodiment. 
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At the end of the signal END at the time to, the content of the memory 1 18 is 
represented by the distribution Ro. The end of the signal END starts the signal ETD whose 
duration is determined by the command generator 313. This signal ETD enables the 
calculation of the range in the memory 1 1 8 of the classifier. 
5 The signals SP (Plus-direction) and SM (Minus-direction) comprised in RTD control, 

respectively, the processing in the positive direction (SP=1) and in the negative direction 
(SM=1) of the range of the distribution Ro that becomes Ri at ti, R2 at t 2 and R 3 at t 3 , etc. 
Thus, the respective durations of SP and SM determine the range of the position of the 
distribution R 5 at the end of the signal ETD. 
10 The multiplexer 207, which is controlled by the command T 5 has two inputs which are 

supplied respectively by the outputs of the expansion and erosion operators, and one output. 
Multiplexer 207 enables implementation of either of these operators in relation to the 
command T. The output of the multiplexer 207 is OUTi : 

15 Oin; = A, xT+B,xT 

Figure 2 1 illustrates a flow chart for anticipation implementation software according 
to one embodiment of the invention. 



XL Time coincidences 

20 In a simplified embodiment described until now, the time coincidences block 102 

comprises a single register contained a single time coincidences value making up the time 
coincidences criterion R. 

A. Complex time coincidences criteria 

25 In a preferred embodiment, the time coincidences block is a memory that may contain 

several values forming together the time coincidences criterion R, any of which is capable of 
enabling the information carried by a pixel. Each of these values is stored in memory in its 
product term register 410 as shown in Figure 22. 

Figure 22 represents a time coincidences block 102 according to one embodiment of 

30 the present invention, which includes a plurality of product terms registers 410 supplied by 
the bus 425 A 'PRODUCT TERM 5 and controlled by the bus Program Register 424. 
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Each of these product term registers 410 has one output that supplies an OR circuit 
421 that provides as output a signal fed into one of the inputs of a controlled inverter 422. 
Inverter 422 receives on its second input the signals from the bus Program Register 424 via 
the register 423. 

5 Figures 23 and 24 illustrate time coincidences block 102 implemented in a Field 

Programmable Gate Area (FPGA) 400. 

Such a memory comprises a controlled inverter 403 whose output is the output of the 
Field Programmable Gate Area 400 and one of whose inputs is connected to an output of an 
OR circuit 401. The inputs of Or circuit 401 are connected to the B lines 405, wherein these 
10 lines intersect the A columns 406 that are connected to amplifiers 402 supplying signals s and 

/• 

The intersections 404 of the lines 405 and of the columns 406 are programmable 
connections enabling to determine the whole operation of the Field Programmable Gate Area 
400. 

15 Figure 24 represents a single line 410 of such a Field Programmable Gate Area 400. 

Such a line 410 comprises registers 411 and 412 for receiving programming variables Reg-aO 
and Reg-bO. This line 410 can be broken down into A elementary function blocks each of 
which comprises a controlled inverter 413, an OR circuit 415 and an inverter 414. One of the 
inputs of the controlled inverter 413 is connected to the corresponding input A and the other 

20 input to the corresponding bit i of the register Reg-aO, where i= 0 to the number of columns. 

The output of controlled inverter 413 is connected to the input of the inverter 414 that 
supplies, through its output, one of the inputs of the OR circuit 415. The other input of OR 
circuit 415 is supplied by the corresponding bit i of the register Reg-bO as shown. The output 
of the controlled inverter 413 also supplies one of the inputs of an n+1 input OR circuit 417 

25 that receives, additionally, all the corresponding signals produced by the remaining inverters 
413 as shown. 

An n+2 input AND circuit 416 whose output is product term 0 receives as one input 
the output of the OR circuit 417 and the outputs of the different elementary functions as the 
remaining inputs. 

30 
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B. The Learning Mode 

The time coincidences block 102 in one embodiment is externally programmed by an 
instruction given by an application management interface (API). This API loads the registers 
411 and 412 of Figure 24 as will be discussed below. 
5 In a preferred embodiment, the histogram calculation unit 1 , in addition to being 

configurable for updating the classifier and for anticipation, is configured for implementing a 
learning function. 

In one embodiment as shown in Figure 25, the histogram calculation unit 1 comprises 
a learning multiplexer 109, which in one mode enables automatic programming of the time 
10 coincidences unit 102. The learning multiplexer 109 selects either of both possible operating 
modes (processing and learning). In the processing mode, the values contained in the register 
of the time coincidences unit 102 are set, and conversely, in the learning mode, these values 
are updated. 

The processing mode 

15 When operating in the processing mode, the learning multiplexer 109 transmits, on its 

output, a signal indicating that the values contained in the registers of the time coincidences 
block 102 are not modified during the operating sequence in processing mode. The values 
stored in these registers have therefore been selected and stored by the user, or they may have 
resulted from a previous learning phase as will be discussed below. 

20 The time coincidences unit 102 also receives, from the other histogram calculation 

units co-operating with that described herewith, comparable signals inE . . . inA. 

Fulfilling its role already described above, this time coincidences unit compares the 
values thus received to the values stored in its register(s) and outputs a signal 102s equal to 1 
in case of coincidence and equal to zero in the reverse case. This enabling signal is sent to the 

25 incrementation enabling unit 107 and when its value is equal to 1, authorises taking into 
account the value of the parameter DATA(A) of the pixel affected in the analysis memory 
100 and, conversely, when the value of the enabling signal is zero, processing proceeds to the 
following next. 

The learning mode 

30 The operation of the histogram calculation unit is controlled by signals represented on 

Figure 5, i.e. an initialisation signal (INIT), a writing signal (WRITE), that carry the 
information corresponding to each pixel in the frame (or the image) and an end signal END. 
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In the learning mode, the learning multiplexer 109 outputs the value of the time 
coincidences signal that is then used instead of DAT A( A). In the learning mode of an i-order 
histogram calculation unit, a signal LEARN enables throughout a frame sequence the 
processing in the learning mode. 
5 During this sequence, the learning registers 116 are updated. Simultaneously, the time 

coincidences block 102 ensures transparency of the signals, enabling the DATA(A), (equal to 
the time coincidences signal 1 1 1) as soon as at least one of the inputs inA, . . ., inE is active 
(=1). 

At the end of the signal WRITE, the histogram memory 100 represents the 
10 distribution of the time coincidences signal. The test unit 103 then generates a classification 
of the occurrences by decreasing value equal in number to B 'SUM TERM'. 

During the signal END, the values of the time coincidences signal thus selected are 
written into the registers 411 and 412 of each block 410 in the time coincidences block 102 
(see Figure 24). The register 412 corresponds to the value of the time coincidences signal 
15 and the register 411 corresponds to its complement. In practice, two outputs of the same 

register can be used, supplying both these values. Thus, automatic statistical elaboration of 
the key-parameters is performed in the frame studied. 

The flow charts of the various software packages necessary to fulfil the self-adapting, 
anticipation and learning functions represented in Figures 6-11, 21 and 23 are self- 
20 explanatory and do not call for any digression to be understood by one skilled in the art. 
When, internally, these Figures refer to variables, the variables have been represented within 
a box. For certain functions that are realised in a particular component described herein, the 
numeric reference of this component has also been allocated to the function. 

25 XII. The spatial and temporal processing unit 

With reference to patent application WO-98/05002, already mentioned above, the 
spatial processing unit 6 preferably outputs various signals including, for example, the signals 
F, SR, V, VL, DI, and CO, each associated with each pixel in a synchronous fashion. These 
are preferably digital signals. The complex signal F comprises a number of output signals 

30 generated by the system preferably including signals outlining the presence and the location 
of a zone or a moving object, V is the velocity of the oriented direction of the displacement 
DI of each pixel in the image. Also, preferably, an output of the system includes the input 
digital video signal which is delayed (SR) in order to synchronise it with the output ZH of the 
frame, while taking into account the calculation time of the compound data signal F (for a 
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frame). The delayed signal SR is used, for example, to represent the image received by the 
camera on a monitor or a television screen that may also be employed to represent the 
information contained in the compound signal ZH. The compound signal ZH can also be 
transmitted to a separate processing unit 10a for further processing. 

5 

XIII. Spatial processing: choice of axes 

The position of a pixel in space is represented with respect to a system of axes. 
According to the shape, the orientation, etc. of the objects in the scene, certain systems of 
axes supply better results than others. 
10 Figures 26 and 27 illustrate the procedure for choosing axes enabling to obtain 

optimised histograms, i.e. exhibiting a clearly defined maximum value. Figure 26 is a 
schematic representation of axis selection circuitry according to one embodiment of the 
present invention, and Figure 27 illustrates various axes selectable by the circuitry of Figure 
26. 

1 5 The Space transform unit 60 receives as input the spatial data x and y that may be 

either Cartesian or polar data. This Space transform unit is controlled by a signal a and, for 
each value of a, outputs a parameter that feeds a histogram constitution unit according to the 
invention. 

The program controlling this histogram calculation unit launched by the Program 
20 Register 424 enables selection of the value a so as to produce an optimised histogram. 

Such a method for selecting the appropriate axes has been described in detail in the 
application PCT WO-98/05002 (see Figure 1 1 and the corresponding description, here the 
'Space Transform 5 unit is referred to as 37), the entire contents of which are hereby 
incorporated by reference for all purposes. 

25 

XIV. Temporal processing 

The colorimetric processing of the values given by the tone (hue), saturation and 
luminance signals, as well as the velocity, direction and intensity signals can be added a 
spatial filtering function that produces a spatial resolution parameter (the Gabor method) and 
30 a binocular function that, via a distance calculation automaton, supplies a depth parameter. 

Complete applications can be realized while processing, in whole or in part, these 
various parameters. 



27 



XV. Visualization of the statistical curve 

Figures 28 and 29 describe more precisely the means for visualising the histogram 
curve. Figure 28 is a schematic representation of a statistical visualisation device according 
to one embodiment of the present invention, and Figure 29 is an example of a result obtained 
5 using the visualisation produced by the device of Figure 28. Figure 8 is a flow chart for 
generating curves and producing overlays according to one embodiment of the present 
invention. 

According to a preferred embodiment, a curve generator 114 enables on screen- 
overlay of a curve of the values DATA for the frame processed previously. Similarly, a 

10 screen overlay of the time coincidences signal is possible. These overlays are sent 

respectively by the lines 14 and 15 to a screen 8. Switches 16 and 17 enable selection of a 
particular histogram calculation unit from among the various histogram processing units. It 
will be apparent that two or more overlays for different histogram calculation units and/or ' 
different parameters can be simultaneously displayed on screen 8. 

15 The memory 100 addressed by the value of the column counter 353 feeds one input of 

a shift register 350 whose other input is supplied by the parameter RMAX generated by the 
analysis register 104. The output of shift register 350 supplies one input of a comparator 351 
whose other input is fed by a row counter 352 via an inverter 354. An AND circuit 355 
receives the result of the comparison P>Q as one input, the variable ValZone as the other 

20 input, and supplies as output the variable Aff_Cbe. 

The column counter 353, which generates the variables c Col_Counter' 356 and 
'Co^Curve^ounter' 357, the row counter 352, which generates the variables 
'Row_Curve_Counter' 358 and 'Row_Counter' 359, and the generator of the variable 
Val Zone constitute a sub-assembly 91 of the sequencer 9. 

25 Moreover, the visualisation control block 365 of the screen 8 receives the delayed 

video signal SR, a cursor command produced by the cursor block 366, and a command 
produced by the semi-graphic memory 367. 

The Figure 29 is the result of the stamp obtained 360 and enabled by the switch 16 
that transfers the curve validation signal to the overlay command 15 n the screen 361 that 

30 comprises moreover a command box 362, a cursor 363 and a text box 364. 

Thus, this screen and the associated mouse constitute a graphic user interface (GUI) 
enabling the user to generate and to command the application. 
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Similarly, the time coincidences function can be visualised, dynamically, in the form 
of pixels 368, by actuating the switch 17 on the time coincidences overlay command 14. 

XVI. Applications 

5 Figure 30 illustrates the implementation of a set of histogram calculation units 1 

enabling the management of any number of parameters A, B, C, D, E. . . Although only 5 
histogram processing units 1 are shown, it will be apparent that any number may be used as is 
necessary. The association of spatial (generally two in number) as well as temporal (at least 
one) parameters enables modelling a spatial-temporal neurone. The temporal processing unit 

10 5 receives the signal S(t) and the CLOCK signal, and spatial processing unit 6 receives the 
CLOCK and synchronisation ST signals (ST) and (SL). 

As represented in Figures 4 and 30, each parameter A, B, C, D, E . . . coming from 
temporal and spatial processing units 5 and 6 feeds a histogram calculation unit, respectively 
1 A , 1b 1e- The time coincidences generated by the set of classifiers 102 is available on the 

15 bus 111 and used as a whole by each of the histogram calculation units, respectively 1 A , 1b 
1e- 

For exemplification purposes, in one embodiment, A, B, and C can represent 
respectively the colour components of the input pixel (e.g., luminance L, tone T and 
saturation S), and D and E can represent the co-ordinates Pi and P 2 of the pixel considered in 
20 an optimised axis system. 

In summary, as represented on Figure 31a, for the parameter A, each histogram 
calculation unit 1 A , 1b,- • •> 1e processes one of the data DATA(A), DATA(B),. . DATA(E) 
by the corresponding function (fog) A . • • to produce individually an output value (1015) A . . . 
and all together, the time coincidence available on the bus 1 1 1 . At the same time, the analysis 
25 output register 104 A is fed. 

The choice of the parameter processed by each histogram calculation unit, the 
contents of the analysis output register 104 and the function fog are determined by the A.P.L 

In the embodiment shown on Figure 31, the different parameters DATA(A) . . . 
DATA(E) feed an input multiplexer 500 that is controlled by a register 501. The register 501 
30 is updated by the command SELECT 502. In one embodiment, a learning multiplexer 503 is 
optionally provided for implementing the learning function as previously described. In this 
embodiment, It is thus possible to use a single histogram calculation unit 1 to process any of 
the different parameters A, B, C . . . E that are addressed by a bus 5 10 in relation to the 
SELECT command 502. The controlled learning multiplexer 503 receives, according to the 
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status of the learning command of the histogram calculation unit i, LEARNi, either the time 
coincidences information transmitted by the bus 1 1 1 , or the information originating from the 
input multiplexer 500. Figure 40 illustrates a functional block diagram of multiple histogram 
calculation units la (e.g., from Figure 31) according to an embodiment of the present 
5 invention. As shown, each histogram calculation unit la is connected to data bus 510 5 which 
provides the various parameters for processing, and to bus 1 1 which provides the 
classification signals 101s and the learning function signals to the various units la. Each 
histogram calculation unit la includes memory 100, classifier 101 and time coincidences unit 
102, and each unit la is capable of implementing the automatic classification, anticipation 

10 and/or learning functionality as previously described herein. It will be apparent that multiple 
histogram calculation units 1 can be operating in the operation mode while one or several of 
the remaining histogram calculation units 1 are operating in the learning mode. 

In one embodiment, a histogram calculation unit is time-shared among different 
parameters during each frame. For example, with reference to Figure 31, histogram 

15 calculation unit 1 according to this embodiment calculates histograms and associated 
statistics for two or more parameters (e.g., Data (A) and Data (C)) during each frame. 
Multiplexer 500, in this embodiment, is capable of time multiplexing the various parameters. 
In this manner, fewer histogram calculation units are needed for processing the desired 
parameters, thereby reducing the amount of silicon required for producing the required 

20 number of histogram calculation units. 

According to the status of the learning command LEARN, The histogram calculation 
unit will operate either in the processing mode or in the learning mode. 

The assembly la thus formed by a histogram calculation unit 1, an input multiplexer 
500, its associated register 501, and optionally a learning multiplexer 503, constitutes a 

25 polyvalent histogram calculation unit. 

Figure 32 represents a complete device comprising, for exemplification purposes, a 
set of sixteen such polyvalent histogram calculation units. These units la constitute a matrix, 
and are connected to a bus 510 on which the parameters D, V, S, T, L, pO, pi, . . pl5 are 
available (pO, pi, p2, . . pi 5 in one embodiment are slopes of reference axes). The bus 111 

30 carries the time coincidences information. In this embodiment, control unit 513 provides 
overall control and determines which of the parameters L, T, S, V, D, pO, pi, . . ., pi 5 are to 
be processed at a given time by one or several dedicated polyvalent histogram unit(s) and by 
the sequencer 9. A processor 520 thus constituted can be integrated on a single solid state 
substrate. The number of polyvalent histogram calculation units 1 a depends on the 
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application and on the solid state components manufacturing technologies available. For 
example, using 0.5 [im technology currently available the integration of 32 histogram 
processing units la is economically feasible. With advances in semiconductor processing 
technology, it becomes possible to fabricate more and more histogram calculation blocks 
5 (e.g., blocks la in Figure 32) on the same chip, and to perform more calculations on more 
samples (i.e., larger and larger numbers of samples per parameter. Such an increase in 
processing capability can be realised without an increase in complexity of the API, which is 
discussed below and illustrated in Appendix A in detail. For example, the same instruction 
set can operate a 20 block device as well as a 200 or a 2000 block device without any added 

10 complexity required. 

In another embodiment, with reference to Figure 39, a processor 605 according to the 
present invention, e.g., similar to processor 520 of Figure 32, is implemented to process 
parameters associated with a perception domain other than the visual perception domain. As 
illustrated in Figure 39, the techniques of the present invention can be applied to analysing 

15 aural, or sound, parameters for applications such as voice recognition and voice-to-text. In 
Figure 39, a sound signal generating device provides sound signals to processor 605, which 
then provides output signals to. In one embodiment signal generating device includes a 
microphone, but it may include any device capable of providing analog or digital signals, for 
example, a CD or DVD player, tape player, etc. Signal generating device preferably provides 

20 digital signals, and may operate in a slave mode or a master mode similar to signal generator 
assembly 2 of Figure 1 . Processor 605 receives the signals and processes various parameters 
of the sound signal. Such parameters include frequency, amplitude and phase. The phase 
and amplitude parameters are analogous to the visual spatial and temporal parameters, 
respectively. Processor 605, provides signals to device 610 so as to enable device 610 to 

25 display desired results. For example, in one embodiment, device 610 includes a printer for 
printing out text associated with signals provided by signal generating device 600. Likewise, 
device 610 may include a monitor or any other text generating device. 

Figure 33 is the representation of a generic visual perception processor 520 (or 530) 
receiving information from a CMOS imaging device 521 including a retina 522 and a 

30 sequencer 523. 

Figure 34 represents a system including a plurality of histogram calculation units la 
capable of operating with several CMOS imaging devices according to one embodiment. For 
example, the association of two CMOS imaging devices 531, 532 represented on Figure 35 
enables acquisition of information on the depth in the scene observed. 
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In certain uses, it is desirable to be able to observe certain shots of a scene, in depth. 
Accordingly, in one embodiment, the retina is fitted with a variable focal device as 
represented on Figure 36. 

Figure 37 is a schematic representation of a system composed of a set of polyvalent 
5 histogram calculation units, capable of processing information originating from three 

directions, respectively VI, V2 and V3 that can represent a three-dimensional space. It is thus 
possible to manage volume perception data, for example, for use in robotics applications. 

XVII. Application program interface (A.P.I.) 
10 The application program interface (A.P.I.) represented on Figure 38 enables to 

provide a complete system including any number of polyvalent histogram calculation units, 
with the set of external parameters that it requires. Its dynamic configuration is thus ensured. 
Appendix A, which is provided as an integral part of this document, includes a functional 
block diagram of the Spatial-temporal API, the graphical user interface (GUI) API, the mouse 
15 API and the I/O API, as well as the various API commands associated therewith, according to 
one embodiment of the present invention. 

Each command mnemonic is associated with an index i corresponding to the number 
of the polyvalent histogram calculation unit for which it is intended. Each mnemonic can be 
accompanied by configuration parameters. Each mnemonic enables allocating the 
20 parameters DATA(A) . . . DATA(E) to real parameters of the scene observed. Certain of the 
commands are as follows: 

SELECT enables to allocate a parameter DATA(A) to a determined unit. 

LEARNi enables to perform the learning function for a polyvalent histogram 
calculation unit i. 

25 START ensures initialisation of a polyvalent histogram calculation unit. This 

command configures the memory 118 of the classifier 101. 

STOP stops the polyvalent histogram calculation unit. It is used as soon as a 
histogram calculation unit is inactive. The overall energy consumption is thereby reduced. 

AFCURV is the curve validation command that controls the switch 16 represented on 
30 Figure 4. Its inverted command is CLCURV. 

AFMAP is the validation command of the time coincidences controlling the switch 
17. Its inverted command is CLMAP. 

MAP is the writing command of the registers 411 and 412 of the time coincidences 
unit 102. 
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MLRN is the command ensuring collection of the content of the time coincidences 
registers 411 and 412 after the learning process. 

These and other commands are explained in more detail in Appendix A. 



5 XVIII Localization 

Figure 41, illustrates a block diagram of a histogram processing unit 1 
according to the present invention. In Figure 41, the parameter DATA(A) is processed by the 
histogram calculation unit 1 by applying the function (f 0 g) 3 thereto. Histogram calculation 
unit 1 comprises an analysis output register reg 4 containing values representing the statistical 
1 0 distribution of the parameter D AT A( A) . 

Histogram processing unit 1 is controlled by the API 5 that determines at the 
same time which parameter, e.g., DATA(A), that the unit processes, the content of the 
register 4 and the function 3 (f 0 g), and supplies the bus 110 with an output value S, which in 
one embodiment, as described above, is a classification value. Histogram calculation unit 1 
1 5 receives as input, from bus 110 the classification values from the other histogram calculation 
units to which it is connected. 

For each of the histogram calculation units, the classification value is 
determined from the application of the function (f Q g) to the sequence of values, e.g., Ay t , of 
the parameter, e.g., DATA(A). The register reg 4 contains the result of various statistical 
20 processes of the sequence, Aj Jt . These processes are defined by the API 5 and their results are 
available for any usage, for example a use defined by the same API 5. 

A preliminary step, to facilitate the processing of the signal, whatever the 
object or its characteristics to be localized, is advantageously provided and includes adapting 
the luminance level and as well in an adaptation of level and of dynamics for localization of 
25 an object in motion. 

Figure 42 illustrates a combination of two histogram processing units 10 and 
1 1 each processing a spatial parameter, X and Y, respectively. The bus 110 supplies the 
histogram calculation units 10 and 1 1 with the information to be processed and an AND 
operator 1 2 combines the output signals S px and S py provided by histogram calculation units 
30 10 and 11, respectively The output of operator 12 is provided to the bus 110. 

Figure 43 is an alternate representation of the system shown in Figure 42. It 
can be easily understood that the set of both histogram calculation units described previously 
and represented in Figure 42 fulfill the same function as one histogram calculation unit 13. 
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In one embodiment, a plurality of histogram calculation units are combined for 
processing a plurality of parameters. For example, as represented in Figure 44, four 
histogram calculation units 16 to 19 each process a parameter. For example, for localisation 
of an object in black and white, the luminance parameter is processed by the histogram 
5 calculation unit 16, the saturation signal is processed by the histogram calculation unit 17, the 
spatial parameter X is processed by the histogram calculation unit 18 and the spatial 
parameter Y is processed by the histogram calculation unit 19. 

Figures 45 and 46 represent a plurality of histogram calculation units that 
provide for the localization of the color of a selected object. The histogram calculation units, 
10 respectively, unit 20 for tone, unit 21 for saturation, and unit 22 for luminance, supply output 
signals combined by the AND operator 23, which outputs a combined feedback signal 
representing the color analyzed. The assembly of Figure 46 represents a function similar to 
that provided by histogram calculation unit 24 represented of Figure 45. 

Figure 47 illustrates a system including two histogram calculation units 1 00 
15 and 101 that process a motion parameter (e.g., speed), and the spatial parameters X and Y, 
respectively. These histogram calculation units 100 and 101 are coupled to each other and to 
the bus 1 10 of the Figures described previously by the line 103. 

The association of histogram calculation units processing, respectively, a 
temporal parameter, e.g., the motion MVT, and spatial parameters such as X and Y, allows 
20 for the localization of a region in space (i, j) in which a point is represented by the parameters 
X, Y for which a sufficient number of points confers the temporal parameter considered (here 
motion), a single value that can be determined directly by the system or predetermined via the 
API. 

The operation of the system of Figure 47 according to one embodiment is as 
25 follows: bus 102 supplies the histogram calculation units 100 and 101 with the data to be 

processed. Assuming that an object is in motion at approximately uniform velocity Vi with 
respect to the background, the histogram generated and processed by the unit 1 00 is in the 
form represented on Figure 49 A. The histogram calculation unit 1 00 supplies via the line 
103, a classification signal Smvt (i, j, t) whose value is 1 for all the pixels i, j whose motion 
30 parameter is close to Vi, more precisely ranging between the benchmarks Vi A and ViB 

determined by the histogram calculation unit 100 in relation to the API. S(i, j, t) takes on the 
value 0 when the motion parameter has a value outside the interval defined by the 
benchmarks mentioned above. 
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Through the line 103, the histogram calculation unit 101 receives this motion 
classification value and therefore only validates the points X, Y for which the motion 
parameter MVT shows a value close to Vi, e.g., Smvt (i, j> t) equals 1 . For this set of points, 
the histogram calculation unit 101 supplies a classification value Sx,y via the line 104, 
5 possibly via the bus 1 10, to the histogram calculation unit 100. 

The line 105 supplies, for any usage, possibly via the bus 110, the value of the 
parameter Vi and the line 106 supplies the classification value designated here ZA, which 
allows for defining a region whose outline 202 represented on Figure 48 is approximately 
representative of the set of points 203 animated by the velocity Vi and its center of gravity 
10 X,,Yl 

The system operates as indicated as long as a sufficient number of points 
determined by the API replies to the system, i.e. for a value of the motion parameter MVT 
ranging between the benchmarks Vi A and ViB. 

When the number of points satisfying this condition is insufficient, the system 
15 is reset, taking into consideration again the set of points of the space 201 and seeking 
therefore a new characteristic region in view of the values of the parameter MVT. 

This operation and this reset are represented by the flowchart of Figure 50. 
When starting (START), the system processes the set of the points of the space 201. If the 
histogram calculation units 100 and 101 isolate a number of points with values characteristics 
20 of the parameter MVT greater than a threshold, the classification signal ZA allows for 
defining a new region of interest. A counter Cr is incremented by one unit for each 
consecutive valid frame. This counter represents the confidence index of the localization 
function. In one embodiment, a new region is declared valid for a value Cr greater than 1 . 

Conversely, when processing a space at a given instant, or still frame, isolation 
25 of a number of points greater than the fixed threshold is generally not possible. In such a 
case, the space region considered (Region of interest) is reset, and the system takes into 
consideration the space in its entirety. The counter Cr is then reset to zero. 

The description made with reference to figures 47 to 50 explains the operation 
resulting from the association of a histogram processing unit that processes a temporal 
30 parameter with a processing unit that processes space parameters. It should be understood 
that in either of these categories, the parameters may be irrelevant. In particular, as regards 
the motion or velocity parameters, luminance, tone or spatial resolution can be processed as 
temporal parameters. 
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Similarly, the space parameters depend on the space considered and on the 
localization system used to define the space. It is thus that in the case of a two-dimensional 
space, different orthogonal orientation marks can be used, wherein the enabling definition of 
the best concentration of points for a given parameter is selected. This process, which 
5 provides for definition of a region, can be localized and repeated in different ways. 

The representation Z designating the classification signal associated with a 
region will be used as well to designate the region. 

Figures 51 and 52 are representations illustrating a system and method for 
determining, from a main region ZA associated with a parameter P, sub-regions Zao, Zai, Za 2 
10 for the same parameter P. Figure 51 illustrates a system including three histogram 

calculation units that, each in relation to criteria defined by the API, allows for processing 
localized sub-regions, respectively Zao, Zai, Za 2 from a region ZA. 
y Thus, as represented on Figure 52, from a general region ZA, the regions Zao, 

m Zai, Za 2 are localized and dissociated. It should be understood that, as represented in Figure 

52, each sub-region Zao can then serve as a main region for extraction of sub-sub-regions 
S Zaoo, Zaoi, Zao 2 ..In general, any region, e.g. sub-region, sub-sub-region, etc., may serve as a 
yg main region for extraction of sub-regions thereto; this process may be repeated as often as 
~ requested by the operator and defined via the API up to a degree of definition or of finesse of 
BP the regions corresponding to the minimum number of points defined to constitute the 

threshold used in the preliminary description with reference to Figure 50. 
rf Figure 53 illustrates a special application of the method and of the device 

described above, specifically a hierarchic analysis of an object, e.g., a human face as shown. 
Histogram calculation units 301 and 300, e.g., similar to the set represented in Figure 47, are 
configured to provide localization of a first region of interest delineated by the outline of the 
25 object in motion, e.g., the "head region". This region is characterized by a dominant color. 

Histogram calculation unit 302 is configured to determine the color variations 
or complements inside this determined region, and to detect new homogeneous regions or 
sub-regions identifying the position of characteristic elements of the object, e.g., the eyes and 
the mouth of the human face. 
30 Each of these sub-regions is then analyzed separately in color, motion, shape, 

to provide specificity and delineation of the face. Each shape is advantageously analyzed 
from the spatial resolution, for example on the basis of the Gabor wave processing, and of the 
orientation. 
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Figure 53 represents the association of a histogram calculation unit 301 that 
process temporal parameters for the motion parameter and a histogram calculation unit 302 
for a tone resulting from the association of the luminance, tone and saturation parameters, 
e.g., as represented in Figures 45 and 46. 

When the association of the histogram calculation units 300 and 301 operating 
as described above does not provide a localization of regions or of sub-regions, because of 
the application of the threshold of numbers of points of the region considered, then the 
histogram calculation unit 302 that processes the parameters LTS relays while substituting in 
its operation to the calculation unit 301 in order to co-operate with the histogram calculation 
unit 300. 

The space histogram calculation units 303 to 305 enable cutting up the main 
region constituted by the frame formed around the outline of the human face into a certain 
number of sub-regions, e.g., formed by the eyes, the mouth, etc. It is thus possible to prepare 
a graph representing the object. This multilevel graph is described by different spatial 
resolutions of the image, wherein the upper level is attributed to the weakest spatial 
resolution and the following levels correspond to increasing spatial resolutions. 

Thus, for example, the head can be defined at the first level by the dominant 
color of the face, of the hair, and of the beard. For each of these characteristics, a greater 
spatial resolution designates regions representing the eyes and the mouth, wherein a third 
level indicates a finer spatial representation of the characteristics of the eyes. 

The description made until now corresponds to the use of a device of the 
invention in order to analyze an object in space whose characteristics are not known initially. 
The device then adapts itself and localizes by itself the characteristic regions. 

It is also possible to supply this device, via the API, with a particular object, 
wherein this description is composed of the definition of a set of regions each corresponding 
to an interval of values of a particular parameter associated with a particular space region. 

From this definition, the device of the invention identifies among the data 
supplied, that corresponding to this definition. It is thus possible to register and to localize a 
pre-set object. 

This predefinition may result from either theoretical considerations, such as 
the relative positions of the centers of gravity of the scatter charts of the regions of interest 
localized, presented by Figure 54, or prior usage of the device during which data 
corresponding to the object was supplied to the device, wherein the device has extracted and 
stored the definition thereof. 
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While the invention has been described by way of example and in terms of the 
specific embodiments, it is to be understood that the invention is not limited to the disclosed 
embodiments. To the contrary, it is intended to cover various modifications and similar 
arrangements as would be apparent to those skilled in the art. Therefore, the scope of the 
appended claims should be accorded the broadest interpretation so as to encompass all such 
modifications and similar arrangements. 
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APPENDIX A 
API Specifications 

4 sub division for GVPP : 

- Spatio-temporal computation API 

- Graphic GUI API 

- Mouse GUI API 

- Communication and input-output API 



DATA 



SELECT 
LEARN 



CLRSCR 



DPDATA 





Spatio-temporal API 




► 




► 

► 












► 

► 
















Graphic GUI API 




► 


► 



DPNTER 
BUTTON 

MVCAM 
GETCAM 
MVMOT 
GETMOT 



Mouse GUI API 



In/Out API 



AFCURV 
CLCURV 
AFMAP 
CLMAP 

GET 
START 
STOP 
MAP 
MLRN 



SENDPC 



GETPC 



Spatio-temporal API Bloc 

This group enables all instructions to run the generic spatio-temporal computations 
and to get the results. 

Functions : 

START : 

Goal: Initialisation of one bloc for the classification. 

Parameter : index bloc, MIN value, MAX value. 

Prototype : 



Bloc3 equ 03 
MIN equ 10 
MAX equ 100 



START Bloc3 MIN MAX 
Input - RO 

R1 

R2 

Output - 



index bloc 
MIN value 
MAX value 



STOP: 

Goal : end of computation. 

Parameter : index bloc. 



Prototype : 



Bloc3 equ 



STOP Bloc3 
Input 
Output - 



RO 



index bloc 



SELECT : 



Goal : Progammation of input parameter bloc(lum, hue, motion, line 

orientation). 

Parameter : Index bloc, type of input parameter. 



Prototype : 



Bloc3 equ 03 
LUM equ 00 



SELECT Bloc3 LUM 



Input - R0 Index bloc 

R1 Input parameter 

Output- 



GET: 

Goal : Get the result computation of one parameter. 

Parameter : Index bloc, Load result parameter. 

Prototype : 



Bloc3 


equ 


03 


MIN 


equ 


00 


MAX 


equ 


01 


RMAX 


equ 


02 


POSRMX 


equ 


03 


POSMOY 


equ 


04 


NBPTS 


equ 


05 



GET Bloc3 NBPTS 

Input - R0 Index bloc 

R1 Index parameter 

Output- R0 result value of this parameter 



LEARN : 
Goal 



Learn the association-context of a bloc 



Parameter : Index bloc. 



Prototype 



Bloc3 



equ 03 



LEARN Bloc3 
Input - R0 
Output- 



Index bloc 



MAP: 
Goal 

Parameter 
Prototype : 



Put on the time coincidences fonction the result of previous learning. 
Index bloc.summ of product-terms. 



Bloc3 



equ 03 



MAP Bloc3 0F3 1AB 007 



Input - 



Output- 



R0 
R1 
R2 
R3 



Index bloc 
First product terms 
Second product terms 
suite 



MLRN : 
Goal 

Parameter 



: Get the result of learning. 
: Index Bloc. 



Prototype : 



MLRN 






Input - 


RO 


Index bloc 


Output- 


RO 


MIN classification 


R1 


MAX Classification 




R2 


First main association (product terms) 




R3 


Second association 




R4 


suite 



AFCURV : 

Goal : Histogram curve drawing of one bloc. 

Parameter : Index Bloc. 

Prototype : 

Bloc3 equ 03 

AFCURV Bloc3 

Input - RO Index bloc 

Output- 



CLCURV : 

Goal : Clear curve of one bloc. 

Parameter : Index Bloc. 

Prototype : 

Bloc3 equ 03 

CLCURV Bloc3 

Input - RO Index bloc 

Output- 



AFMAP : 

Goal : Learning Bloc drawing. 

Parameter : Index Bloc. 

Prototype : 

Bloc3 equ 03 

AFMAP Bloc3 

Input - RO Index bloc 

Output- 



* 



CLMAP : 

Goal : Clear the learning bloc drawing. 

Parameter : Index Bloc. 

Prototype : 

Bloc3 equ 03 

CLMAP Bloc3 

Input - R0 Index bloc 

Output- 



Graphic GUI API 
CLRSCR : 

Goal : Clear Screen. 

Parameter : No. 

Prototype : 

CLRSCR 
Input - 
Output- 



DPDATA : 
Goal 

Parameter 
Prototype : 



: Display ASCII code on screen. 

: ASCII code, row position, column position. 



DPDATA 
Input - 



Output- 



R0 
R1 
R2 



ASCII code 
row position 
column position 



Mouse GUI API 
DPNTER : 

Goal : Mouve and display the pointer. 

Parameter : row position, column position. 



Prototype : 



DPNTER 
Input - 

Output- 



R0 
R1 



row position 
column position 



BUTTON : 

Goal : get the action of button. 

Parameter : Button. 

Prototype : 

BUTTON 
Input - 

Output- RO new position of buttons 



API E/S 



MVCAM : 
Goal 

Parameter 



Move the camera. 

X Position, Y Position, Focus. 



Prototype 



MVCAM 
Input - 



Output- 



RO 
R1 
R2 



X position 
Y position 
Focus 



GETCAM : 
Goal 

Parameter 



: Get the camera's parameters. 
: No. 



Prototype 



GETCAM 
Input - 
Output- 



RO 
R1 
R2 



X position 
Y position 
Focus 



MVMOT : 
Goal 

Parameter 



: Action motor. 
: Sens+steps. 



Prototype 



MVCAM 

Input - RO 

Output- 



Sens+steps 



f 



If 

GETMOT : 

Goal : Get the actual position of motor. 

Parameter : No. 

Prototype : 

GETMOT 
Input - 

Output- RO position 

SENDPC : 

Goal : Send one information to the PC. 

Parameter : information pointer. 

Prototype : 

SENDPC 

Input - RO information pointer 

Output- 



GETPC : 

Goal : Get an information from PC. 

Parameter : No. 

Prototype : 

GETPC 
Input - 

Output- RO information 



